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Preamble 


Everyone agrees school leadership is important. Research suggests that there 
are substantial (albeit mostly indirect) relationships between school leadership 
and student achievement. Over the past 40 years, the field has developed and 
tested various hypotheses about the leader characteristics, skills, knowledge, 
and behaviors that may be relevant to improving student outcomes. However, it 
is challenging for practitioners and policymakers to determine whether and how 
school leaders are making a difference in student achievement. 


Through its Education Research Grants Program (84.305A), the Institute of 
Education Sciences (IES) has tried to address this challenge by inviting research 
applications on school leadership, but the number of applications reviewed is low 
relative to other topics, and many important questions remain unanswered. IES 
hopes to expand its portfolio of research on the influence of school leaders on 
student outcomes. Therefore, during this Technical Working Group meeting, IES 
asked participants for their input on how to encourage research that can help 


e Advance theory and build evidence on the relationships between 
leadership and student achievement; 
e Operationalize these models into leadership standards or leadership roles; 


e Assess the extent to which leaders meet these standards or fulfill these 
roles; 


e Develop and evaluate professional development for leaders; and 


e Determine the extent to which school leaders impact student achievement. 
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Meeting Summary 


Welcome 
The meeting began at 8:40 a.m. 


Dr. Thomas Brock, Commissioner of the National Center for Education 
Research (NCER), welcomed the participants and expressed enthusiasm for the 
meeting topic, which he said urgently needed more attention in the field. Dr. 
Brock noted that he himself had done some investigation into educational 
leadership in community colleges when he worked for MDRC (formerly 
Manpower Demonstration Research Corporation) on “Achieving the Dream,” a 
national initiative designed to help community colleges make better use of data to 
improve instruction and services. 


Dr. Brock noted that there is strong interest in school leadership—both in the 
federal government and at the state level. NCER has funded a few influential 
studies on the subject, but would like to fund more. Part of the problem is that not 
enough fundable proposals have been submitted. 


One difficulty for research studies on education leadership is how to measure 
and assess leadership. Finding answers to that question is complicated by 
politics. In addition to measurement issues, Dr. Brock pointed to the difficulty in 
connecting quality of school leadership to student outcomes. Finally, there is the 
difficulty posed by there being too few individuals trained to conduct research on 
education leadership. He wondered whether there might be opportunities to 
interest researchers working in business or management fields in this topic. 


Dr. Brock stated he wanted to hear people’s thoughts about where high-quality 
research is being done and where research is not being done but should be. 
Does IES need to offer different kinds of opportunities? He urged participants to 
be frank. While IES’s budget is not unlimited, he expects it to remain level in the 
immediate future. He mentioned that IES can also identify priorities through its 
Research and Development Centers and Research Networks. As examples, Dr. 
Brock pointed to the Standards and Schools Research and Development Center, 
started last year, as well as the College Completion and Early Learning Research 
Networks. 


Dr. Katina Stapleton, Program Officer for the Improving Education Systems 
research portfolio, outlined the following two purposes of the meeting: 


e To identify ways to encourage more research on education leadership to 
be undertaken in the field; and 
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e To identify ways to measure what education leaders do that can help 
students. 


Dr. Stapleton then gave an overview of the IES-funded grants related to 
education leadership. These projects have been funded through several IES 
research grant programs since 2004, but are currently being funded through the 
Improving Education Systems research program within the Education Research 


grant program for FY 2016. 


IES’s investment in research on education leadership includes the following: 


e The development and validation of leadership assessments; 


e The development of interventions that will help principals and other school 
personnel become better leaders; 


e The exploration of relationships between school leaders’ knowledge/ 
skills/behaviors and student achievement; and 


e The evaluation of interventions for school principals. 


Dr. Stapleton then described two other potential IES funding streams for 
research on education leadership that she believes are under-utilized. The first is 
NCER’s Researcher—Practitioner Partnerships in Education Research grant 
program, in which researchers and education agencies partner to conduct 
research that has direct implications for improving programs, processes, 
practices, or policies that may result in improved student outcomes. The second 
is the National Center for Special Education Research’s Special Education 
Policy, Finance, and Systems research program, which also funds research on 
systems-level practices and policies. She invited any of the participants who 
might have expertise in research partnerships or special education to present 
ideas during the course of the meeting. 


Finally, Dr. Stapleton discussed the need for research on leaders other than 
principals (e.g., assistant principals, leaders at the district or regional level). She 
also noted that the current funding timeframes for NCER-funded studies of 
school leadership make it difficult for researchers to collect and analyze student 
outcomes before the end of the grant. 
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Panel 1: Framing the Issue 
Robert Marzano and Jacqueline Wilson 


Dr. Marzano, CEO of Marzano Research, provided a broad overview of research 
challenges that have emerged in education since his seminal studies of school 
and district leadership. 


Dr. Marzano noted that lack of reliable measures of effective leadership at the 
school- and district-level inhibits research on education leadership. He noted that 
nomenclature, defining complex behaviors, and differences in opinion about what 
concepts mean make it difficult to interpret and compare the results of studies. 
For example, one study might conclude that effective school leaders exhibit 
“visibility,” while a second study might use a different term to describe the same 
underlying “visibility” construct, and a third study might use the term “visibility” but 
mean something that is conceptually different from how it is used in the first 
study. In effect, it is difficult, if not impossible, to determine the construct 
underlying many of the independent variables regarding leadership 
characteristics in extant literature. 


Other challenges Dr. Marzano identified included the need for the field to 


e Identify the most important functions of school leaders in order to help 
leaders prioritize their efforts; 


e Explore the distribution of leadership roles/responsibilities within schools; 
and 


e Identify what kind of supports (such as professional learning communities) 
can help guide the leadership process. 


Dr. Wilson, Assistant Professor in the School of Education at the University of 
Delaware and Director of the Delaware Academy for School Leadership, 
discussed the “refresh” of the Interstate School Leaders Licensure Consortium 
(ISLLC) standards. Dr. Wilson jointly chaired the 2014 ISLLC refresh committee 
with Joseph Murphy. 


According to Dr. Wilson, the first set of standards, published in 1996, 
emphasized instructional leadership. In 2008, the National Policy Board for 
Educational Administration (NPBEA), along with the Council of Chief State 
School Officers (CCSSO), issued a new version of the standards that changed 
the emphasis to leaders’ functions—i.e., what leaders are expected to do. Dr. 
Wilson discussed an ongoing debate about how many standards are really 
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necessary (and are they the right ones). A new committee has been formed to 
review the proposed 2015 standards.’ 


PANEL 1 DISCUSSION HIGHLIGHTS 


Following the panel presentation, a group discussion centered on the following 
themes related to leadership standards: 


The degree to which ISLCC standards are policy based and the quality of 
research supporting the new professional standards (2015); 


The need to empirically test assumptions that leadership standards are 
based on; and 


The way in which leadership standards are being used and the degree to 
which the use of standards actually leads to improvement. 


MAIN RECOMMENDATIONS OF PANEL 1 


More and better research is needed on principal leadership and how to 
help principals make changes in their practice. In particular, IES should 
support research into whether evaluating principals based on leadership 
standards works to improve principal retention or other desirable 
leadership and student outcomes. 


IES should challenge researchers to investigate the validity of the 
underlying theories of leadership that leadership standards are based on. 


IES should support research into what standards are currently being used 
to assess principal performance and how schools and school districts deal 
with leadership standards. 


Research is needed to inform the improvement of measures of practice 
and to develop more sensitive measures of near-term outcomes of 
leadership and leadership interventions. 


' Subsequently, in November 2015, the Professional Standards for Educational Leaders 2015 were published. 
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Panel 2: Testing Theoretical Models Through Exploratory 


Research 
Jason Grissom and Elaine Allensworth 


Dr. Grissom, Associate Professor of Public Policy and Education in the 
Department of Leadership, Policy & Organizations at Vanderbilt University, 
discussed his IES-funded exploratory study conducted with Susanna Loeb, which 
aimed to identify the specific attributes, skills, orientations, and behaviors of 
school leaders that are associated with positive school outcomes. 


The research team estimated the relationship(s) between school-leader 
characteristics and school outcomes (including teacher turnover, school climate, 
and student achievement). The team initially worked with four urban districts 
(some dropped out), using the following data collection techniques: 


e Annual surveys of principals, vice principals, and teachers; 
e Interviews with principals; and 


e Full-day observations of principals. Over the life of the project, they 
captured 750 days of principal practice. 


Their major findings include the following: 


e Principals’ skills in organizational management had a large and consistent 
association with measures of school outcomes, much larger than 
associations with other skill sets, such as management of instruction. 


e Overall time spent by principals on instructional issues correlated 
negatively with student achievement growth, though some specific 
categories of instructional time, such as coaching of teachers, correlated 
positively. 


e Results from one study raised substantial concerns about the validity of 
the use of student test scores to measure principal performance. 


e Teacher collaboration was associated with positive student outcomes, 
suggesting that principal support for teacher collaboration is a promising 
strategy for school improvement. 


Dr. Grissom noted that given these findings, it is important to question the current 
tendency to privilege principals’ role as instructional leader over other functions 
(such as organizational management). 


Dr. Allensworth, Director of the University of Chicago Consortium on Chicago 
School Research and Managing Director of the University of Chicago Urban 
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Education Institute, said that her IES-funded study of school leaders in the 
Chicago public schools produced an answer to the question of what effective 
principals do. Specifically, effective principals foster a climate that promotes 
safety/order and sets academic expectations. In comparing schools that made 
improvements with schools that did not, her study found that, on the whole, the 
high-achieving schools had a strong safety climate. In turn, her study found that 
schools with stronger climates were those in which teachers had considerable 
power over decision-making about safety issues. 


Overall, her study found that effective principals had 


e Clear, coherent goals that were articulated in similar ways among all 
respondents in the school, rather than many unconnected goals; 


e A focus on students (not on teachers or curricula); 


e A focus on monitoring student data around the goals with teacher teams; 
and 


e Systems and strategies for achieving goals that provided universal 
supports (i.e., rather than leaving it to individual teachers). 


PANEL 2 DISCUSSION HIGHLIGHTS 


Following the panel presentation, group discussion focused on the following 
issues: 


e It can be difficult to determine how long it takes for a good principal to 
make a difference, especially in low-achieving schools. Current research 
does not provide a convincing time estimate for assessing leadership 
intervention impacts. A handful of studies point to 4 or 5 years as a marker 
for measuring results, which is challenging given standard IES funding 
timelines. 


e Acculture of safety is essential for student improvement. Many questions 
remain around safety factors, such as: What is the priority for school 
leadership? What should a principal do first in an unsafe school? Should 
he or she impose discipline, make sure that teachers impose order, get 
behavior coaches to help students change their behavior? 


e Causes of principal turnover and retention warrant further study, including 
frustration with politics, especially with regard to school boards, and the 
influence of federal policies such as Race to the Top. 


e Trying to discern leadership impact is challenging when there are so many 
potentially confounding variables in schools. 
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MAIN RECOMMENDATIONS OF PANEL 2 


e Further research should be undertaken to assess the effectiveness of 
instructional leadership compared to the effectiveness of other things that 
principals do (such as creating safe school environments or managing 
school resources). 


e Research should be funded on the effects of principal retention on student 
achievement, as well as on the reasons for non-retention (conflicts with 
school boards, policies such as Race to the Top, etc.). 


e More studies should be undertaken to test the provisional conclusion that 
a safe and orderly school environment promotes improved student 
achievement and that promoting such an environment is something that 
effective principals do. 


e IES should continue to recommend that grantees share their study 
findings broadly and in ways that are readily accessible by practitioners 
and policymakers. 


e IES could provide reviewers with guidance on content features to look for 
in a strong leadership study. 


Panel 3: Evaluating School Leaders 
Matthew Clifford, Ellen Goldring, Richard Halverson, and Elizabeth Warner 


Dr. Clifford, Principal Researcher in the Education Program of the American 
Institutes for Research (AIR), oversees a portfolio of research and evaluation 
studies and provides services to the field around principal evaluation and 
coaching. He currently oversees three studies, looking at principal professional 
development, principal preparation, and principal evaluation. 


Dr. Clifford reported on a study completed in 2013 for the Center on Great 
Teachers and Leaders, a national content center funded by the U.S. Department 
of Education’s Office of Elementary and Secondary Education. At that time, 50 
states and 2 territories had adopted new educational evaluation policies, to 
assess both teachers and principals. Forty-eight states have adopted the ISLLC 
standards, although they customize them to their particular needs. The new 
policies reemphasize the need for annual evaluations. Most states allow school 
districts 2 years to design and launch new evaluation systems. With funding from 
the Wallace Foundation, Dr. Clifford and colleagues are updating the policy 
analysis and have launched an interactive website on policy standards at 


www.principalstandards.gtlcenter.org. 
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There are relatively few models for principal evaluation. States are currently 
selecting from among the following four models: 


e The local-control approach—districts develop their own models aligned 
with district/regional education priorities and values (e.g., Vermont). 
Reliability and uniformity may be sacrificed with this approach. 


e The single evaluation model—the model is adopted statewide, with very 
little latitude, which improves reliability and uniformity but allows for little 
local adaptation to local/regional educational priorities (e.g., Mississippi, 
US Virgin Islands, Tennessee). 


e The cafeteria approach—the state offers four to five models for districts to 
pick from (e.g., New York). 


e State models plus local control—the state suggests four to five models, 
but local districts can request a waiver and use their own evaluation 
system (e.g., Idaho and Maine). 


Most state principal performance evaluation systems include practice measures 
and outcome measures. Practice measures may include observation and rating 
of principal feedback to teachers, principal completion of annual professional 
growth plans, and principal portfolios. 


Although all states have passed new legislation, and many states are piloting 
new principal evaluation systems, few states are studying how well their principal 
evaluation systems perform. A scan of leadership research by Dr. Clifford and 
colleagues identified eight studies that examined principal evaluation systems’ 
quality and impact. All of the identified studies examined client satisfaction with 
system implementation, but none examined impact on principal practice 
improvement using independent measures, principal retention/mobility, improved 
school culture, or improved student learning. Further, he noted that few states 
have taken steps to validate principal evaluation frameworks and instruments (an 
example is Pennsylvania), and few states have publicly reported statistics on 
inter-rater agreement, rater severity, or other standard measures of personnel 
evaluation system performance. 


Dr. Clifford described the following challenges to designing effective principal 
evaluations: 


e Identifying sound and predictive measures, particularly for principal 
observation and 360-degree surveys. Few measures are proven to be 
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valid and reliable. An exception is the Vanderbilt Assessment of 
Leadership in Education (see below). 


e Differentiating accurately between evaluation measures. Unfortunately, 
the typical results don’t show that sufficient distinctions were made 
between school years, don’t reveal much about the qualities of the 
principal under consideration, and so forth. 


e Executing studies of principal evaluation system performance remains 
challenging for states, given limited availability of state-level funding for 
research, evaluation system implementation timelines, system scale-up 
strategies, and the relatively low number of principals. 


e States also remain challenged to maintain focus on principal evaluation 
systems development, given the pressure to evaluate teachers. 


Dr. Goldring, Chair of the Department of Leadership, Policy and Organizations, 
Peabody College, at Vanderbilt University discussed the Vanderbilt Assessment 
of Leadership in Education (VAL-ED) project, which was funded by the Wallace 
Foundation and IES. VAL-ED is an evidenced-based rating scale that assesses 
principals’ behaviors known to directly influence teachers’ performance and, in 
turn, students’ learning. The assessments incorporate input from all members of 
the school’s professional community: the principal rates him- or herself; the 
teachers rate the principal; and the principal’s supervisors rate the principal. This 
is a 360-degree methodology in which the findings lie in the “gaps” between the 
various ratings. The rating process is administered online. 


Dr. Goldring noted the tepid response coming from the scientific, practice, and 
policy fields regarding the uses and consequences of principal evaluation data 
and suggested that there are other data-based, objective ways to evaluate 
principals than evaluations or judgments by teachers, supervisors, and self- 
feedback/reflection by principals. For example: 


e Principals could receive evaluation around the quality of newly hired 
teachers by tracking the effectiveness of new teachers over time. 


e Principals could be evaluated based on the percentage of ineffective 
teachers who improve and grow, given reliable and valid teacher 
evaluation scores. 


e Principals could be evaluated based on the alignment between 
professional development support and teachers’ areas of growth, such as 
through examination of professional development plans and professional 
development engagement. 
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e Principals could be evaluated in terms of how effective they are at 
retaining highly effective teachers. 


Dr. Goldring pointed out the lack of consensus in defining effective principal 
evaluation indicators, underscoring the need for more study. 


Dr. Halverson, Professor in the Wisconsin Center for Education Research at the 
University of Wisconsin-Madison, offered his thoughts on the future of leadership 
evaluation. He has worked on developing the Comprehensive Assessment of 
Leadership for Learning (CALL) survey, also supported by IES. 


The CALL survey, used in over 300 schools in the United States and abroad, 
raises questions about what school leaders do and how their work should be 
guided. Rather than focusing on an individual leader, such as the principal, CALL 
measures the tasks carried out by various individuals across the school. The 
survey uses a 360-degree methodology. All administrators, teachers, and other 
instructional staff in the school take the CALL survey, and schools receive a 
customized plan detailing the key tasks for improvement. Dr. Halverson 
explained that CALL is not really an evaluation instrument, but it can provide a 
baseline measure of coaching-based improvement. 


Dr. Halverson encouraged the participants to consider alternative models for 
thinking about education leadership. As an example of how this can be done, he 
described an alternate learning model that resides outside of school and in which 
learners (including adult learners) join “affinity spaces.” Those who participate in 
affinity spaces form “distribution networks,” and then expand to “discourse 
communities,” and thereby grow the field. Such communities are familiar to 
online gaming fans, for example, but could be relevant to an academic 
environment as well, and this alternate approach could address the issue of 
whether students’ performance on tests has any bearing on how well they adapt 
to their lives beyond school. 


Dr. Halverson would like to see research that explores the question: “What does 
education leadership look like in the context of affinity space-based learning?” 


Dr. Warner, Economist in the Evaluation Division of the National Center for 
Education Evaluation and Regional Assistance, discussed the following three 
investigations being conducted by IES that are focused wholly or in part on 
evaluation of principals: 


e Impact Evaluation of Support for Principals (2014-2019); 


e Impact Evaluation of Teacher and Leader Performance Evaluation 
Systems (2011-2017); and 
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e Impact Evaluation of the Teacher Incentive Fund (2009-2017). 


In the Impact Evaluation of Support for Principals, a randomized controlled trial 
(RCT) taking place in 100 schools will provide evidence on the effectiveness of a 


principal professional development (PD) program and its ability to improve 
principals’ leadership skills and school quality. The treatment condition includes a 
heavy emphasis on instructional leadership activities, such as school 
walkthroughs and classroom observations, accompanied by constructive 
feedback. The ultimate goal of the principal’s support will be facilitating teacher 
growth and improving student achievement. The program targets high-poverty 
districts. The first year’s report is due in spring 2018. 


The Impact Evaluation of Teacher and Leader Performance Evaluation Systems 
is designed to examine the implementation and impacts of a package of 


performance evaluation system components, including measures of student 
achievement growth, classroom observations, and measures of principal 
leadership. Teachers, leaders, and districts have received constructive feedback 
on teacher and principal performance. Principals in the approximately 15 study 
schools within each of the 8 participating districts received feedback on their 
performance based on VAL-ED, which was administered twice each study year. 
A report of the study’s first year is expected by spring 2016. 


The Impact Evaluation of the Teacher Incentive Fund (TIF) is investigating the 


effect of performance-based pay on high-performing teachers in low-performing 
schools with high-need students. One of the study questions is, “How do 
teachers and principals in schools that did or did not offer pay-for-performance 
bonuses compare on key criteria?” Although the first year’s report was issued 
before the first actual pay-for-performance payouts were distributed, some 
information was already available. Preliminary findings indicated that educators 
in schools that offered pay-for-performance bonuses tended to be less satisfied 
overall than those in schools that did not offer such bonuses. However, 
educators in schools offering pay-for-performance bonuses were satisfied 
specifically with the opportunity to earn additional pay, even though a greater 
percentage indicated they felt pressured to perform due to the TIF program. 


PANEL 3 DISCUSSION HIGHLIGHTS 
Following the panel presentation, the group discussed the following topics: 


e Whether all students can succeed in the types of personalized, 
community-based learning environments described by Dr. Halverson (for 
example, ones involving discourse communities or ones built on student 
choice). Questions include: What is the teacher’s or principal’s role in that 
environment? How would they be evaluated? 
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e How to find schools that are interested in trying new approaches and 
willing to be part of implementation research. Questions include: Could 
there be a registry or matching process facilitated by IES so it is not left up 
to the researchers to recruit schools? What types of incentives (e.g., 
federal grants) might be appropriate for schools to be research sties? 


e How IES could attract more innovative leadership researchers to the 
funding process, especially given a lack of response in the past. 


MAIN RECOMMENDATIONS OF PANEL 3 


e Fund more studies to learn what forms of principal evaluation are most 
effective and how well evaluation systems perform overall. 


e Encourage more studies on principal practice as well as validation of 
measures of leadership quality. 


e Encourage researchers to partner with states on proposals to examine 
principal evaluation system implementation and impact. 


e Fund research on education leadership that integrates discovery models 
of learning and design. 


Panel 4: Developing and Evaluating Professional Development 
for School Leaders 
Beatriz Ceja, Cheryl King, and Roger Goddard 


Ms. Ceja, Program Manager of the School Leadership Program at the U.S. 
Department of Education’s Office of Innovation and Improvement (OIll), began by 
describing the discretionary grant program that supports the development, 
enhancement, or expansion of innovative programs to recruit, train, and mentor 
principals (including assistant principals) for high-need local education agencies. 


More than 60 grants have been funded in this program, at different funding 
levels, between 2002 and 2013. In some cases, the same organization has been 
funded more than once. Collecting data on the program is difficult because the 
funded groups have focused on different aspects of professional development 
(PD) for principals (e.g., sometimes coaching is included, though it is unclear 
what kind of coaching and how much coaching was involved), and the cohorts 
have been small. 


Some of the questions that have arisen from grantees concerning principal 
professional development pertain to rigor, intensity, and duration. For example, 
should first-year principals receive the same level of PD as fifth-year principals? 
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What PD is appropriate for principals who move from one school to another 
within the same district? For principals who are participants in the School 
Leadership Program and move to a different district not partnered with the grant, 
should they continue to receive grant-funded PD? 


Ms. Ceja indicated that, although she believes that coaching of principals is very 
helpful, it has been difficult to consolidate results and, therefore, difficult to make 
an argument for more funding for this area of research. The impact of the 
services and preparation of principals has been difficult to capture. Additionally, 
research that meets the What Works Clearinghouse standards regarding PD for 
principals or principal preparation is scarce. Consequently, using evidence-based 
criteria for the School Leadership Program created various obstacles such that it 
was made an invitational priority. This resulted in only seven grantees 
responding to the invitational priority, and of the seven, only two submitted 
findings that contained some element of evidence or validation. 


Ms. Ceja also noted that new allowances for many discretionary programs to add 
2 additional years for grantees to collect additional data on the effectiveness of 
the interventions being implemented through the initial grant could allow for 
greater understanding of the program’s impact. Without data, it’s difficult to know 
if one intervention worked better than the other. One problem, she stated, is that 
applicants tend to feel that receiving federal funds to implement an intervention is 
proof that the approach works. Some applicants seem reluctant to follow up with 
an actual evaluation component. 


Dr. King, Principal Investigator and Director of Quality Measures for Principal 
Preparation at the Education Development Center in Waltham, MA, spoke on the 
subject of principal preparation. Her comments were based on lessons learned in 
over a decade of work with preparation program providers, states, and large 
urban school districts. Using Quality Measures™ Tools and Protocols, to 
facilitate preparation program reviews, Dr. King and her team work with training 
program providers and school districts to assess the extent to which preparation 
programs are having an impact on school leader performance. This work has 
been principally funded by the Wallace Foundation and IES (through the National 
Research and Development Center on Scaling Up Effective Schools). 


The primary lessons learned included the following: 


e Strengthening and sustaining partnerships between districts and principal 
training providers, based on local context, is essential to ensure the 
development and growth of effective school leaders. 


e Evidence-based approaches to program self-assessment are particularly 
useful in developing shared understandings of effective practices and 
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identifying areas of focus for a joint reform agenda between school 
districts and preparation program providers. 


e Research-based interventions implemented and studied for the purpose of 
increasing the impact of school leaders on school, teacher, and student 
performance must become a priority. 


e The extent to which training programs collect and use principal 
performance data from school districts during the first 3 years of 
induction—and whether such data are used to determine the types of 
programmatic changes needed—is another area requiring further study. 


At some point, Dr. King would like to evidentially link teacher and principal 
training to school, teacher, and student performance. She stated that the 
outcomes of preparation and training seem successful at the moment, but the 
evidence is not well developed. 


She expressed regret that many principals are not followed after completion of 
their formal training programs. Dr. King believes that the preparation programs 
should follow the principals they have graduated to determine whether their 
training is having an impact. More broadly, there is a need to examine to what 
extent training programs collect and use performance data from school districts 
to guide preparation practices. 


Dr. Goddard, Novice G. Fawcett Chair of Educational Administration in the 
College of Education and Human Ecology at The Ohio State University, 
discussed his IES-funded experiment on the efficacy of the Balanced Leadership 
Professional Development Program for School Leaders. The study involved 127 
elementary schools in Northern Michigan. The schools were rural and poor. The 
study asked both teachers and principals about principals’ leadership knowledge 
and behavior, school climate, and teachers’ collaborative and instructional 
practices. The study examined the causal impact of the professional 
development program for school leaders on teacher and principal perceptions of 
school climate and instruction, educator turnover, and student achievement. 


Findings revealed discrepancies between teachers’ and principals’ perception in 
many areas. For example, while treatment school principals believed their 
leadership knowledge and skill had increased, teachers did not see growth in 
these areas. Similarly, while treatment principals generally reported 
improvements in the climates of their schools, their teachers did not. One area 
where treatment principals did not report substantively significant growth, 
however, was in their own involvement in curriculum, instruction, and 
assessment. Thus, while principals worked with other principals during the PD 
sessions, it may be that they did not work with their own teachers differently or in 
ways that teachers noticed as different based on the training. This may be why 
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principals said they loved the training they received and that it gave them more 
confidence while at the same time it made no difference to student achievement. 
Thus, the benefits of the principal training do not appear to have filtered down 
enough to the teachers to have impacted students. In addition, while principals 
who received the training were significantly less likely to leave their positions 
than those who did not—and teachers in schools where principals received the 
training were also less likely to leave during the time of the study—these 
reductions in participant turnover did not translate into significant learning gains 
for students. Dr. Goddard regarded these findings as reflective of potential 
shortcomings in the design of the PD. 


Dr. Goddard also noted that it was necessary to keep the principals in the control 
schools from receiving the PD training to avoid contamination in the experiment. 
Treatment school principals may have interpreted this as reason to not involve 
their teachers. He regards this as a disadvantage of the study design. Other 
possible reasons Dr. Goddard noted for the lack of impact on student 
achievement included the observation of one case study treatment principal that 
he/she needed more central office support and another who noted that he/she 
could not “do it alone.” 


A notable finding of the study was based on a quasi-experimental study that did 
not evaluate the PD but rather examined the conditions that predicted student 
learning in the study schools prior to the beginning of the PD. The study showed 
that the greater the instructional leadership teachers reported in their principals, 
the greater the degree to which teachers reported collaborating frequently on 
instructional improvement. Teacher collaboration, in turn, was a significant 
predictor of the level of collective efficacy beliefs in schools, which in turn 
positively and significantly predicted differences among schools in 3% grade 
students’ mathematics and reading achievement. 


PANEL 4 DISCUSSION HIGHLIGHTS 


Following the panel presentation, the group discussed several issues and raised 
the following questions: 


e Over the years, Oll’s School Leadership program did not require that 
grantees evaluate the effectiveness of their interventions, making it difficult 
for grantees to present evidence-based results to show that coaching 
made a difference in PD. Could the requirement to present evidence be 
written into the grant solicitation? Could the funded grants themselves be 
viewed as an opportunity to produce evidence? 


e Are potential applicants to Oll and IES grant programs interested in 
rigorous evaluation, and what is the right amount of grant money to attract 
more applicants to the programs? 
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e Little is known about assistant principals. 


e The degree to which PD matters to principal success. Are principals ready 
at the start to handle the challenges of their position? What interventions 
could be provided, if needed? 


MAIN RECOMMENDATIONS OF PANEL 4 


e Research should examine whether principal training programs are 
producing a pipeline of high-performing principals who meet or exceed 
school district expectations and whether principal impact on school, 
teacher, and student performance is linked to preparation and training. 


e IES could fund studies that would evaluate the success of principal 
training programs. 


e IES should encourage use of similar measures across leadership studies 
to facilitate comparisons and conclusions regarding what works and what 
does not work. 


e IES should encourage applicants to study larger samples and involve 
more states and school districts to increase generalizability. 


e IES should fund more studies of the effects/effectiveness of principal 
coaching. 


Final Thoughts From TWG Members 


At the end of the meeting, each participant was asked to give suggestions about 
how IES can support more research on school leadership. 


Dr. Marzano suggested that perhaps principals should change every 2 to 3 
years, as is done in the military with those in leadership positions. 


Dr. Wilson expressed concern about punitive performance evaluation systems 
and cautioned that soon there won't be any leaders/principals left. Instead, the 
emphasis should be on professional growth. 


Dr. Grissom emphasized the need for longer grant periods, for more human 
capital (e.g., more people working in this area), and for greater emphasis on 
measurement (e.g., the establishment of a National Center for Leadership 
Performance and Assessment). 
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Dr. Allensworth also noted a preference for longer grant periods so that student 
outcomes can be studied in connection with principal performance. 


Dr. Clifford noted that opening up more measures of achievement was a good 
idea, particularly when the validated measures offer opportunities to examine 

near-term impact of leadership interventions. Additionally, he believes that we 
should invest in developing better measures of principal practice quality. He is 
also concerned that we know very little about how principals are selected and 
hired and about principal workforce attrition/mobility. 


Dr. Goldring stated that a lack of resources makes it difficult to focus on student 
achievement. She suggested that IES drop the link to student achievement for 
now—given that grants have relatively short time periods and that it is, therefore, 
difficult to establish a connection between principal performance and student 
outcomes—and break studies down into smaller components to test even basic 
assumptions. 


Dr. Halverson maintained that outcomes should not be linked to test scores nor 
used as a way of measuring success. He pleaded for more flexibility in 
interpreting outcomes. 


Ms. Ceja agreed that the pool of qualified and available principals (from the pool 
of assistant principals) is shrinking. She also noted that there are still not many 
applicants for OIl’s School Leadership grants. 


Dr. King indicated that it is essential to link principal performance and student 
outcomes. She thought IES’s partnership grants would be an attractive source of 
funding for studying principal performance. 


Dr. Goddard indicated that emphasis should be on measures of leadership 
training that affects teacher practice (instructional change), which would require a 
longer study period. 


Wrap-Up 


Dr. Brock concluded the meeting by saying that the Institute would be examining 
the notes from the day’s discussion as it begins work on the FY2016 Requests 
for Applications (RFAs). He noted that some recommendations may require more 
thought or take time to implement, while others could be made in the near future. 
Finally, Dr. Brock thanked the participants and emphasized that the Institute is 
open to participants’ ideas at any time. 


The meeting adjourned at 4:15 p.m. 
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